X-MimeOLE: Produced By Microsoft Exchange V6.5
Received: by onstor-exch02.onstor.net 
	id <01C78C4E.BF721CFA@onstor-exch02.onstor.net>; Tue, 1 May 2007 17:13:45 -0700
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C78C4E.BF721CFA"
Subject: RE: FS/EVM large I/O support + inverted timeout fixes
Date: Tue, 1 May 2007 17:13:36 -0700
Message-ID: <BB375AF679D4A34E9CA8DFA650E2B04E037975EE@onstor-exch02.onstor.net>
In-Reply-To: <BB375AF679D4A34E9CA8DFA650E2B04E037973C2@onstor-exch02.onstor.net>
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
Thread-Topic: FS/EVM large I/O support + inverted timeout fixes
Thread-Index: AceLgM3/IfbgR1gwR/OlgUPQ69MIRgAmVOUgAAB8T3AAAL78UAAAgH9g
References: <BB375AF679D4A34E9CA8DFA650E2B04E0379708F@onstor-exch02.onstor.net> <BB375AF679D4A34E9CA8DFA650E2B04E03797393@onstor-exch02.onstor.net> <BB375AF679D4A34E9CA8DFA650E2B04E037973BD@onstor-exch02.onstor.net> <BB375AF679D4A34E9CA8DFA650E2B04E037973C2@onstor-exch02.onstor.net>
From: "Maxim Kozlovsky" <maxim.kozlovsky@onstor.com>
To: "Jonathan Goldick" <jonathan.goldick@onstor.com>,
	"dl-Design Review" <dl-designreview@onstor.com>

This is a multi-part message in MIME format.

------_=_NextPart_001_01C78C4E.BF721CFA
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable




_____________________________________________
From: Jonathan Goldick=20
Sent: Tuesday, May 01, 2007 11:09 AM
To: Maxim Kozlovsky; dl-Cougar
Subject: RE: FS/EVM large I/O support + inverted timeout fixes

1.	In 5.1.3, we may want to change the read behavior described.  If
you get a read in the middle of a file (Spec workload) it's not a great
idea to do a large read starting from that point.  This approach would
likely hurt spec performance, but could be change somewhat to in fact
improve performance.  Does the change you suggest have no impact on
dcache?  At the least we only support a dcache covering 64B, with
alignment rules.  Will that change as well?
[MK] Why it is not a great idea to do large read? We have to read this
data no matter what and the fastest method is to read it all at once
without moving the disk heads around.
JG> Reading data you don't need ejects something out of the cache that
you might use.  Spec as a workload will exercise that problem by design.
JG> You didn't answer the question about dcache changes.
[MK]=20

The wording in the document is confusing. The intended behavior is that
we never request more data than dcache asks for, but within the
requested range we use the smallest number of EVM requests. No dcache
changes are planned.

2.	In 5.1.4, do you have any guidelines worked out for the various
bobcat modules on how many we will support concurrently?  At the least
we need to spec it so that throughput and Spec do not decrease on these
models.
[MK] The same as we support now. All bobcats have the same FC memory
configurations, so the number of concurrent requests does not depend on
the model.
JG> OK, but I don't know what that limit might be so thought I would
ask.
[MK] The current limit is 2K meta requests and 4k user requests on 2260.
(2240 the limit is 2k+2K, on 2220 it is 2k + 1k) . This is 51MB of FC
buffer space + <6MB of per request internal data.
3.	Need a section on limits for the filesystem tune choices.
[MK] With the eee descriptor chains the limit on the largest I/O
supported is kind of arbitrary. We should make that arbitrary limit at
reasonably low number that will cover 90% of cases just to limit the
amount of testing we need to do, the exact number is to be determined.=20
JG> I'm not sure that's true what dcache is taken into account, but I
was thinking more about the tunables like logwriteperiod and
logwritesize.

[MK] The log IO settings represent tradeoff between the throughput and
response time. I can not say what the values are going to be at this
moment, we will need to do testing with different workloads to
understand the impact of changing these parameters.

This should go to dl-designreview as a rule.

_____________________________________________
From: Maxim Kozlovsky=20
Sent: Monday, April 30, 2007 4:40 PM
To: dl-Cougar
Subject: FS/EVM large I/O support + inverted timeout fixes

Here is the brief description of the changes in the file system and EVM
which are going to be done as part of cougar to help us achieve the
performance goals and also fix some insanity in the current code. While
cougar is the reason for the changes, it is entirely possible that we
should ship this with Zonda to continue the Delorean performance
improvements. The fixes for the inverted timeouts and log replay
optimizations should help us to increase the reliability and
availability which is the Zonda goal.

Please send me the comments; we'll schedule the review meeting if
necessary.

Max

 << File: evm.doc >>=20

------_=_NextPart_001_01C78C4E.BF721CFA
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.5.7651.59">
<TITLE>RE: FS/EVM large I/O support + inverted timeout fixes</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Jonathan Goldick<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Tuesday, May 01, 2007 =
11:09 AM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></B></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> Maxim Kozlovsky; =
dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> RE: FS/EVM large I/O support + inverted timeout =
fixes</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">1.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">In 5.1.3, we may want to change the read =
behavior described.&nbsp; If you get a read in the middle of a file =
(Spec workload) it&#8217;s not a great idea to do a large read starting =
from that point.&nbsp; This approach would likely hurt spec performance, =
but could be change somewhat to in fact improve performance.&nbsp; Does =
the change you suggest have no impact on dcache?&nbsp; At the least we =
only support a dcache covering 64B, with alignment rules.&nbsp; Will =
that change as well?</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">[MK] Why it is not a great idea to do large read? We have =
to read this data no matter what and the fastest method is to read it =
all at once without moving the disk heads =
around.</FONT></I></B></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">JG&gt; Reading data you =
don&#8217;t need ejects something out of the cache that you might =
use.&nbsp; Spec as a workload will exercise that problem by =
design.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">JG&gt; You didn&#8217;t answer the question about dcache =
changes.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">[MK]</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I> =
</I></B></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">The wording in the document is =
confusing. The intended behavior is that we never request more data than =
dcache asks for, but within the requested range we use the smallest =
number of EVM requests. No dcache changes are planned.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">2.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">In 5.1.4, do you have any guidelines worked out =
for the various bobcat modules on how many we will support =
concurrently?&nbsp; At the least we need to spec it so that throughput =
and Spec do not decrease on these models.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">[MK] The same as we support now. All bobcats have the =
same FC memory configurations, so the number of concurrent requests does =
not depend on the model.</FONT></I></B></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">JG&gt; OK, but I don&#8217;t know what that limit might =
be so thought I would ask.</FONT></I></B></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">[MK]</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I> <FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">The current limit is 2K meta =
requests and 4k user request</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">s on =
2260.</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"> (</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">2240 =
t</FONT></I></B></SPAN><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">he =
limit is 2k+2K, on 2220 it is 2k + 1k)</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial"></FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN LANG=3D"en-us"><B><I> <FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">. This is 51MB of FC buffer =
space + &lt;6MB of per request internal data.</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I></I></B></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">3.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</FONT><B><I></I></B></SPA=
N><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">Need a section on limits for =
the filesystem tune choices.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">[MK] With the eee descriptor chains the limit on the =
largest I/O supported is kind of arbitrary. We should make that =
arbitrary limit at reasonably low number that will cover 90% of cases =
just to limit the amount of testing we need to do, the exact number is =
to be determined. </FONT></I></B></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
COLOR=3D"#000080" SIZE=3D2 FACE=3D"Arial">JG&gt; I&#8217;m not sure =
that&#8217;s true what dcache is taken into account, but I was thinking =
more about the tunables like</FONT></SPAN><SPAN LANG=3D"en-us"> <FONT =
FACE=3D"Courier New">logwriteperiod and logwritesize.</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><B><I></I></B></SPAN><SPAN =
LANG=3D"en-us"><B><I><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">[MK]</FONT></I></B></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">The log IO settings</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">represent</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"> tradeoff between the throughput and response =
time. I can not say what the</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"> <FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial">values</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"> are going to be at this moment, we will need to =
do testing with different workloads</FONT></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" =
SIZE=3D2 FACE=3D"Arial"> to understand the impact of changing these =
parameters.</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT COLOR=3D"#000080" SIZE=3D2 =
FACE=3D"Arial">This should go to dl-designreview as a =
rule.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">_____________________________________________<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B></B></SPAN><B><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">From:</FONT></SPAN></B><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> Maxim Kozlovsky<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B></B></SPAN><B><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">Sent:</FONT></SPAN></B><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> Monday, April 30, 2007 4:40 PM<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B></B></SPAN><B><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">To:</FONT></SPAN></B><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma"> dl-Cougar<BR>
</FONT></SPAN><SPAN LANG=3D"en-us"><B></B></SPAN><SPAN =
LANG=3D"en-us"><B></B></SPAN><B><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Tahoma">Subject:</FONT></SPAN></B><SPAN =
LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"></SPAN><SPAN =
LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Tahoma"> FS/EVM large I/O support =
+ inverted timeout fixes</FONT></SPAN><SPAN LANG=3D"en-us"></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"></SPAN><SPAN LANG=3D"en-us"><FONT =
SIZE=3D2 FACE=3D"Arial">Here is the brief description of the changes in =
the file system and EVM which are going to be done as part of cougar to =
help us achieve the performance goals and also fix some insanity in the =
current code. While cougar is the reason for the changes, it is entirely =
possible that we should ship this with Zonda to continue the Delorean =
performance improvements. The fixes for the inverted timeouts and log =
replay optimizations should help us to increase the reliability and =
availability which is the Zonda goal.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 FACE=3D"Arial">Please =
send me the comments; we&#8217;ll schedule the review meeting if =
necessary.</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us"><FONT SIZE=3D2 =
FACE=3D"Arial">Max</FONT></SPAN></P>

<P DIR=3DLTR><SPAN LANG=3D"en-us">&nbsp;&lt;&lt; File: evm.doc =
&gt;&gt;</SPAN><SPAN LANG=3D"en-us"> </SPAN></P>

</BODY>
</HTML>
------_=_NextPart_001_01C78C4E.BF721CFA--
